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Executive Summary 


BACKGROUND: CHILDREN’S PHYSICAL 
ACTIVITY AND THE COVID-19 PANDEMIC 


Physical activity confers myriad benefits for 
children’s physical, emotional, and academic 
health and well-being, but fewer than half of 
U.S. children achieve the recommended 60 
minutes of moderate-to-vigorous physical 
activity each day. Children of color and 
children with socioeconomic disadvantage 
may face particular barriers to physical 
activity; however, school-based programs 

may help reduce such disparities, especially 
given that nearly all children in the United 
States attend school. Early evidence has shown 
that, amid widespread school closures, the 
COVID-19 pandemic may have decreased 
children’s physical activity levels, and these 
decreases may be notably pronounced for 
children of color. This, in turn, may lead to 
increased disparities in other physical activity- 
related outcomes, like pediatric obesity. 


RESEARCH CONTEXT, METHODS AND AIMS 


The Boston Foundation has supported several 
physical activity programs operating in Boston 
Public Schools: Build Our Kids Success; 
Boston Run Clubs; Community Rowing, 

Inc.; Playworks; and Sportsmen's Tennis and 
Enrichment Center. Initiated in 2019, this 
research effort aimed to advance under- 
standing of the reach and effectiveness 

of these programs for participating children. 


Our original research plan, involving in-person 
measurements of physical activity among 
program participants using accelerometry, 
became infeasible due to the COVID-19 
pandemic. However, we were able to pivot to 
an alternative evaluation approach that enabled 
us to evaluate these programs by analyzing 
secondary data and collecting original data 
through remote methods. Secondary analysis 
included data from Boston Public Schools (BPS) 
about physical activity—related school practices 
and environments in BPS as well as publicly 
available school demographic data. Original 
data included surveys of physical activity 
program leadership and front-line staff, as 

well as key informant interviews with 
program leadership. Finally, because the 

other components of this study could not 
include Boston Run Clubs (an initiative that 
ended in 2019), we conducted a systematic 
review of peer-reviewed papers testing 

the effect of similar running programs 

on physical activity and fitness levels. 


Using these data, we aimed to understand: 

(1) how many and what types of schools and 
children were reached by Boston Foundation-— 
funded physical activity programs? (2) what 
dose of physical activity programming time did 
those programs provide and how effective was 
that programming? (3) how did the COVID-19 
pandemic affect program implementation 

and effectiveness? and (4) how did 

support provided by the Boston 

Foundation impact programs? 


KEY RESULTS 


Several important themes emerged in Fall 2019 to around 13 schools and 2,700 


children participating per program on average 


from across our research methods. children in Fall 2020. The dose of programming 


Programs’ Reach (e.g., weeks of programming, minutes per 


ided for th hildren that 
Boston Foundation—-funded physical activity ee ee 


ined dalso d harpl 
programs achieved wide adoption and reach ee ee 


in BPS: In 2016-2020, the programs reached 
34 schools and more than 5,600 children per 


during this time. That being said, programs 
did develop a variety of programmatic 
innovations (e.g., new remote delivery models 
and take-home toolkits) that enabled them 

to continue providing some programming 


year, on average, and that reach was equitable 
based on building-level demographics. 


However, the programs tended to cluster in through the pandemic, and many of those 
certain buildings, particularly those withmore — innovations may continue and enhance 
supportive physical activity environments in programming in the future. Although coaches 
general, suggesting a potential opportunity generally agreed that they were able to deliver 
to reach more schools with an express need. remote lessons successfully, few agreed that 
for new physical activity opportunities. programming during the pandemic was as 


iia as Bicec and ease eanne: effective at getting kids physically active as 
§ : . it would have been without the pandemic. 

In general, the programs provide a meaningful ; 
; . Encouragingly, program leaders reported 
programming dose, averaging about 27 weeks/ rae a ’ potential 

optimism about programs’ poten 

year, 4 sessions/week, and 1.5 hours/session ; P b d ie : h 8 A P . a 
from 2016 to 2020. Qualitatively, program pode nr aewee nentieeie ee 


leaders reported that benefits of programming Impact of Programs’ Relationship 
include not only increases in physical activity with the Boston Foundation 

but also benefits for social-emotional well- Qualitatively, program leaders reported 
being, school attendance, and academic various benefits of their funding from 
success. Preliminary results from our and broader relationship with the Boston 
systematic review provide some evidence Foundation. They emphasized, for example, 
that youth running programs may also that the funding enabled their programs 
help to increase physical activity and cardio- to expand their overall reach, especially to 
respiratory fitness among participants. schools and children with socioeconomic 


i . Further, th Iti- 
Impact of the COVID-19 disadvantage. Further, the multi-year 


Pandemic on Programming commitment from the Foundation and 


funding flexibility provided critical stabilit 
Program leaders reported that the pandemic : 6 : P m 
; . in general, and particularly through the 
led to substantial funding losses and staff cuts. ; . ; 
; : ; pandemic, and the broader relationship 
This may have contributed to the decrease in . ; 
: with the Boston Foundation pushed them 
the average number of schools and children i ee i. 
o be more strategic and innovative. 
served, from around 30 schools and 4,400 5 


CONCLUSIONS 


This mixed-methods evaluation provides 
evidence that these physical activity programs 
achieved broad, equitable reach in Boston 
Public Schools and provided a meaningful 


amount of programming time for participating 


children. However, these programs tend to 
cluster in certain buildings, particularly in 
ones with otherwise supportive physical 
activity environments. These programs 

may provide benefits beyond physical 
activity in areas like social-emotional health 
and academic achievement. Although the 
pandemic substantially decreased nearly 

all dimensions of program reach and dose, 
the programs developed some innovations, 
like remote delivery, that enabled them to 
continue to serve children; these innovations 
may persist into the future. Program 

leaders believe that the Boston Foundation 
played an important role in programs’ 

reach, especially to schools with limited 
resources, and also that the flexibility of 
Boston Foundation funding was crucial to 
organizational health through the pandemic. 


These programs 
may provide 
benefits beyond 
physical activity in 
areas like social- 
emotional health 
and academic 
achievement. 


Background 


United States 
guidelines suggest 
that children and 

adolescents 
should engage 
in 60 minutes 


or more of 
MVPA daily. 


Physical activity (PA) offers numerous benefits 
for children and adolescents.’ For example, 
evidence indicates that regular moderate-to- 
vigorous physical activity (MVPA) supports 
students’ cognitive function, academic perform- 
ance, and mental health’ along with numerous 
dimensions of cardiometabolic health.* 


United States guidelines suggest that children 
and adolescents should engage in 60 minutes 
or more of MVPA daily.* However, objective 
measurements of children’s activity generally 
show that youth do not meet these guidelines. 
For example, the most recent objectively 
measured, nationally representative data on 
PA in the United States showed that just 42% 
of children and 8% of adolescents met PA 


recommendations.° 


Research has shown that students of color have 
less access to safe, accessible outdoor spaces 
compared with their white counterparts.° 
However, given that nearly all school-aged 
children in the U.S. attend school, school- 
based PA programming can help to reduce 
disparities in PA opportunities.”* 


Preliminary evidence suggests that the 
COVID-19 pandemic and associated school 
closures may reduce children’s PA levels and 
exacerbate disparities.’ Schools began to close 
their doors in March 2020, impacting access 
to PA opportunities (e.g., physical education, 
recess, PA programs) that schools often 
provide." The effects of the pandemic and 
school closures have disproportionately 
impacted certain subgroups, including 


children with socioeconomic disadvantage." 
For example, urban school districts maintained 
remote schooling for longer durations than 
did non-urban districts.* Even within the same 
district, students of color were more likely than 
white students to experience longer durations 
of remote schooling.’” These patterns may, in 
turn, worsen disparities in PA opportunities. 


Research capturing the uptake of remote 

PA programming during the pandemic 
shows minimal engagement. In one study, 
parents reported that only 10.4% of children 
participated in team sports training sessions 
through remote/streaming services;’* 28.9% 
participated in virtual martial arts, yoga, 

or dance classes; and 2.4% participated in 
streaming classes or sessions provided by 
acommunity gym.” Limited participation 
in remote PA opportunities may have 
implications for total PA levels. In the same 
study, 36% of parents reported their child had 
done much less PA in the past seven days than 
before the pandemic, and 41% of parents 
reported their child had done much more 
sitting in the past seven days than before the 
pandemic." Another study reported MVPA 
decreased by 12 minutes per day compared. 
to pre-pandemic levels, with high schoolers 
having the most substantial decreases."* 


These findings may have implications for 
disparities in PA-related health outcomes. 
For example, a U.S. microsimulation study 
estimated that decreased PA levels caused by 
COVID-19 may worsen the childhood obesity 
prevalence rate overall, especially for non- 
Hispanic Black and Hispanic children.” 

One model estimated increases in childhood 
obesity prevalence of 2.9% and 3.1% in 
Hispanic and Black children, respectively, 
compared with 2.0% among their White 
counterparts. 


Even before the COVID-19 pandemic, public 
health recommendations emphasized the 
critical role schools play in providing equitable 
opportunities for PA." As schools return to 
in-person classes, school administration 

and teachers may need to play an even more 
significant role providing students opportun- 
ities for PA to reverse PA decreases and health 
inequities intensified by the COVID-19 
pandemic. 


41% of parents 
reported their 
child had done 
much more sitting 
in the past seven 
days than before 
the pandemic. 


Overall Evaluation Approach and Methods 


The Evaluating Children’s Physical Activity 
in School-Based Programs study, led by 
researchers at ChildObesity180 at the Tufts 
Friedman School of Nutrition, was originally 
funded by the Boston Foundation in 2019 to 
assess PA programs in Boston Public Schools 
(BPS) that were part of the Foundation’s 
portfolio. Our original conceptualization of the 
study was to use gold-standard methodologies 
(e.g., accelerometry) to evaluate children’s PA 
during these programs and to understand 
whether these programs were equitably 
benefitting children along dimensions 

like race-ethnicity, weight status, sex, and 
socioeconomic status. In Spring 2020, our 
research protocol was fully developed 

and approved by both the Tufts University 
Institutional Review Board (IRB) and the BPS 
Office of Data and Accountability; however, 
due to school closures and restrictions on 
human subjects research, that protocol was 
not possible to execute. 


Therefore, in 2020-2021, we developed an 
alternative protocol that would be feasible 

to administer remotely. This new protocol 
centered mainly on evaluating four school- 
based programs funded by the Boston 
Foundation: Build Our Kids Success (BOKS), 
Community Rowing, Inc. (CRI), Playworks, 
and Sportsmen’s Tennis and Enrichment 
Center (Sportsmen’s). Overall, this evaluation 
addressed the following key questions, using 
a mix of quantitative and qualitative methods: 


@ How many schools adopted Boston 
Foundation-funded PA programs, and what 
were the characteristics of those schools in 
terms of building-level demographics and 
PA-supportive policies and practices? 


@ How many children were reached by the 
programs? 


@ What dose of PA programming time did 
the programs provide? 


@ How did the COVID-19 pandemic affect 
programs’ implementation and 
effectiveness? 


e@ How did support provided by the Boston 
Foundation impact the programs? 


Our evaluation included analysis of two 
secondary data sets: 


e@ BPS Health & Wellness Policy Report Profiles 
Data: BPS Health & Wellness provided 
deidentified data collected through their 
biannual wellness policy report process for 
2017-2018 (the most recent data available), 
including information on PA-related 
environmental factors, such as the presence 
of PA programs, the amount of PA time 
provided for students, and the presence of 
policies that support student PA. 


@ Massachusetts Department of Elementary 
and Secondary Education Data: We 
extracted publicly available, building- 
level demographic data for all Boston 
Public Schools, enabling us to examine 
whether the availability of Boston 
Foundation-funded PA programming and 
other PA-related policies/practices differed 
by building-level student demographics. 


Our evaluation also included three types of 
original data we collected in early 2021: 


e@ PA Program Staff Surveys: This online 
survey of adult program staff who 
implement PA programing gauged 
perceptions of program implementation 
processes, minutes of PA time provided, 
and the impact of COVID-19 on program 
delivery. 


PA Program Leadership Surveys: This online 
survey of senior leaders from each PA 
partner program gauged the number of 
buildings adopting the programs, student 
reach, number of program staff, and 

the dose of programming customarily 
delivered, as well as the impact of the 
COVID-19 pandemic on these factors. 


Key Informant Interviews: Remote interviews 
conducted with senior leaders from each 
program provided additional qualitative 
insights in areas such as program reach, 
effectiveness, and implementation 
processes; the impact of COVID-19 on 
program operations; and the impact of 
Boston Foundation funding. 
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Finally, because we were unable to otherwise 
include substantive evaluation of BPS run clubs 
(for which Boston Foundation funding ended 
in 2019), we also initiated a systematic review of 
peer-reviewed papers exploring the impact of 
running programs on children’s PA and fitness. 
This review provides context for the potential 
impact Boston Foundation-funded run clubs 
may have had on children and will also 
advance science through new published 
research. 


Secondary Data Analysis 


OVERVIEW 


This analysis includes two secondary data 
sources: (1) BPS School Profiles data include 
survey data about numerous dimensions of the 
school physical activity environment in BPS 
buildings, as reported by school administrators 
and physical educators in 2017-18 (the most 
recent year or which data were available); and (2) 
publicly available, building-level demographic 
data from the Massachusetts Department of 
Elementary and Secondary Education (DESE), 
including the percentage of students with 
socioeconomic disadvantage, percentage of 
students who are of color, and percentage of 
students whose first language is not English. 


These data were analyzed in four domains: 


1. Demographic and PA-related characteristics 


3. Associations between demographics of 
Boston public schools and adoption of 
Boston Foundation-funded PA programs 


4. Associations between PA-related policies 
and practices and adoption of Boston 
Foundation—funded PA programs 


Demographic and Physical Activity- 
Related Characteristics of All Boston 
Public Schools 

To contextualize the subsequent analyses, 
Table 1 describes the overall building-level 
demographics of all Boston public schools, 
including indicators of student economic 
status and student race-ethnicity for the 
district overall and for elementary versus high 
schools. Overall, the district mostly serves 
children experiencing economic disadvantage 


of all Boston Public Schools and children of color. 
2. Adoption of Boston Foundation-funded 
PA programs in BPS: number of schools 
adopting programs and level of within- 
building clustering 
TABLE 1 


SCHOOL-LEVEL DEMOGRAPHICS OF BOSTON PUBLIC SCHOOLS 


ELEMENTARY AND 
CHARACTERISTIC RENE MIDDLE SCHOOLS =e 
(ON Etor)) 
% of students with economic disadvantage* 
(median, IOR) 71.2 (19.6) 69.2 (24.3) 73.6 (9.7) 
% of students of color (median, IQR) 92.7 (16.1) 89.8 (17.1) 95.9 (8.8) 
Total enrollment (median, IQR) 385 (327) 365 (309) 430 (374) 


Source: Department of Elementary and Secondary Education 


*Defined based on whether students participated in one or more of the following programs: the Supplemental Nutrition Assistance Program (SNAP); the 
Transitional Assistance for Families with Dependent Children (TAFDC); the Department of Children and Families’ foster care program; and MassHealth 


(Medicaid). 


Table 2 describes PA-related policies and schools more conducive to PA by expanding 


practices in BPS. Overall, elementary and opportunities before, during, and after school 
middle schools tend to provide more weekly and promoting more universal adoption of 
minutes of PA time than do high schools, policies known to support children’s PA (e.g,, 


prohibiting the withholding of PA as 


driven largely by more minutes of recess. 


There are numerous opportunities to make punishment). 
TABLE 2 
PHYSICAL ACTIVITY-RELATED POLICIES AND PRACTICES IN BOSTON PUBLIC SCHOOLS 
CHARACTERISTIC Rete || criiviegerie | es 
(ON tsi) 
Mean # of TBF-Funded PA Programs (Mean, SD) 1.0 (1.2) 122) 0.4 (0.8) 
Mean total # of PA Programs (Mean, SD) 1.7 (1.6) IO) 1.0 (1.2) 
Minutes of weekly PE (Median, IQR) 66.3 (52.9) 60.0 (40.0) 110.0 (118.3) 
ae Feprovided weedy Oude EE | 1764 (es) 184.1 (105.4) 98.6 (180.0) 
Minutes of weekly recess (Median, IOR) 100.0 (84.3) 100.4 (42.3) 0.0 (121.3) 
Have Comprehensive School PA Plan (% yes)** 5e)9) 64.2 47.6 
Have principal who encourages activity breaks (% yes) 82.4 87.3 69.0 
Have budget for equipment/curriculum (% yes) 60.5 65.5 58.8 
Have PE or PA professional development (% yes) 85.6 89.0 75.6 
Have gym facilities (% yes) SiL.3} 50.6 Sisks) 
Supportive PA Environment Score** (mean, std) 10.8 (3.8) 11.6 (2.8) 8.4 (3.8) 
Certified PE teacher (% Yes) 86.4 ODF 66.7 
Provide in-class activity breaks (% yes) 82.9 96.3 44.8 
Provide before-school PA (% yes) 57.4 67.1 31.0 
Provide after-school PA (% yes) 73.0 68.3 86.2 
i transportation liaison to promote active travel 50.9 54.2 aa 
Prohibit withholding PA as punishment (% yes) 50.0 54.2 O79 
Have indoor PA spaces (% yes) 76.5 78.8 70.0 
Have outdoor PA spaces (% yes) 50.4 Beye 30.0 


Data Source: BPS Profiles Data 

*Sum total of school-reported minutes provided for recess, classroom movement breaks, active classroom lessons, and PA-based promotional activities. 
Minutes were reported by grade level; values here are the mean across grade levels. 

**Composite score for the overall extent to which the school supports student PA. This modified version of the PASS Score” includes points for number of 
physical education (PE) minutes required, licensure for PE teachers, minutes of recess provided, provision of playground supervisors, opportunities for 
PA before/after school and in class, presence of a transportation liaison to promote active transport, policies prohibiting withholding PA as punishment, 
and presence of indoor and outdoor PA facilities /spaces. Range: 0-20, with higher scores denoting more supportive environments. 

See Appendix D for more detail. 


TBF-funded = Boston Foundation-funded 
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Adoption of Boston Foundation- 
Funded PA Programs in BPS 

Table 3 shows the number of Boston Public 
Schools that had zero, one, two, three, and four 
of the Boston Foundation-funded PA programs 
during 2017-18, as reported by lead physical 
education teachers. This reflects some clustering 
of programs within individual buildings: 
Nearly half (47%) of Boston public schools 

had no Boston Foundation-funded programs, 
whereas 15% had three or more programs. 


TABLE 3 


CLUSTERING OF TBF-FUNDED PROGRAMS IN 
BOSTON PUBLIC SCHOOLS, 2017-18 


# TBF-FUNDED PROGRAMS | #scHools 


0 54 
1 25 
2 19 
3 14 
4 3 


Data Source: BPS Profiles Data 
TBF-funded = Boston Foundation-funded 


Demographics of Boston Public Schools 
and Adoption of Boston Foundation- 
Funded PA Programs 

Table 4 shows average building-level 
demographic characteristics of schools that 
did versus did not have each of the Boston 
Foundation-funded PA programs in 2016-17. 
Analyses tested whether the schools where 
these programs are adopted are different from 
non-adopting schools in terms of level of 
economic disadvantage among students, the 
proportion of students who are children of 
color, or the proportion of students who whose 
first language is not English. 


Overall, we found little evidence of 
associations between these building-level 
demographic factors and adoption of the PA 
programs. In other words, the schools reached 
by these programs were demographically 
similar to the rest of the district. The one 
exception was for Sportsmen’s Tennis: 39% of 
Sportsmen's schools disproportionately serve 
students who do not speak English as a first 
language, compared with 49% of schools that 
did not have Sportsmen’s programming. The 
magnitude of this difference is small, however. 


TABLE 4 


ASSOCIATIONS BETWEEN BUILDING-LEVEL STUDENT DEMOGRAPHICS IN BOSTON PUBLIC SCHOOLS 
AND ADOPTION OF TBF- oe a ACTIVITY PROGRAMS, 2017-2018 


DEMOGRAPHIC CATEGORY 


SPORTSMEN’S PLAYWORKS 


Mean % students with 
economic disadvantage 


65.8 74 | 0.58 | 64.1 


YES NO YES NO YES NO YES NO 
(N=44) | (N=72) (N=33) | (N=82) (N=16) | (N=99) (N=25) | (N=90) 


0.28 | 64.3 7.2 | 0.48 | 66.5 | 66.8 | 0.94 


Mean % students of color 86.2 | 86.8 | 0.82 | 85.7 


86.9 | 0.71 | 82.5 | 872 | 0.31 | 84.6 | 871 | 0.46 


Mean % students whose first 
language is not English 


Source: BPS Profiles Data, DESE Data 


46.5 | 48.2 | 0.62 | 46.9 


477 | 0.83 | 39.0 | 48.9 | 0.03* | 46.5 | 478 | 0.76 


p-values are for results of independent-samples t-tests of differences between means 


t Statistically significant at p < 0.05 


PA-Related Policies and Practices 

and Adoption of Boston Foundation- 
Funded PA Programs 

We also explored differences in PA-related 
school policies and practices based on the 
extent to which schools were adopting Boston 
Foundation-funded programs. These analyses 
aim to gauge whether the PA programs are 
reaching schools that otherwise have 
supportive PA environments. 


Table 5 compares Supportive PA Environment 
Scores and minutes of weekly PA provided for 
schools that do versus those that do not have 
each PA program. We found that each of the 
four programs tended to be in buildings that 
had higher overall scores for supportive PA 
environments. However, given the cross- 
sectional nature of this data, it is difficult to 
determine causal direction—i.e., whether the 


PA programs helped to make schools more 
supportive of PA, or if schools that were more 
supportive of PA tended to be more likely to 
adopt PA programs. Whatever the reason, this 
does show that these programs tend to have 
less presence in buildings with less general 
support for PA. 


Conversely, there were no significant 
differences in the average number of PA 
minutes provided across recess, classroom 
breaks, active classroom lessons, and PA-based 
promotional activities. However, this may be 
because, given the secondary data available to 
us, the times incorporated into this total did 
not include before or after school (when BOKS, 
CRI, and Sportsmen’s typically operate), and 
Playworks focuses more on increasing the 
quality of recess than on increasing recess 
duration. 


TABLE 5 

MEAN SUPPORTIVE PA ENVIRONMENT SCORES AND MEAN MINUTES OF SCHOOL PA PROVIDED WEEKLY 

(IN RECESS, CLASSROOM BREAKS, ACTIVE LESSONS, AND PROMOTIONAL ACTIVITIES) IN SCHOOLS THAT 
HAVE VERSUS SCHOOLS THAT DO a HAVE TBF-FUNDED PA PROGRAMS, 2017-2018 


DEMOGRAPHIC CATEGORY 


Mean Supportive PA 


Environment Score* Ue: 


9.8 |0.0001* 


NO YES NO 
(te) (N=16) | (N=99) 


12.4 


PLAYWORKS 


NO 


10.4 


SPORTSMEN’S 


0.004" 


10.2 | 0.001) 12.4 | 10.5 | 0.02* | 12.4 


Mean Min. of School PA 
Provided (Weekly)** 


212.0 | 193.7 | 0.47 | 184.1 


207.2 | 0.34 | 173.1 | 205.2 | 0.26 | 197.7 | 201.4 | 0.88 


Source: BPS Profiles Data, DESE Data 


*Composite score for the overall extent to which the school supports student PA. This modified version of the PASS Score17 includes points for number 
of physical education (PE) minutes required, licensure for PE teachers, minutes of recess provided, provision of playground supervisors, opportunities 
for PA before/after school and in class, presence of a transportation liaison to promote active transport, policies prohibiting withholding PA as 
punishment, and presence of indoor and outdoor PA facilities /spaces. Range: 0-20, with higher scores denoting more supportive environments. 


See Appendix D for more detail. 


** Sum total of self-reported minutes provided for recess, classroom movement breaks, active classroom lessons, and PA-based promotional activities. 
Minutes were reported by grade level; values here are the mean across grade levels. 


p-values are for results of independent-samples t-tests of differences between means. Due to skewness, the values for mean minutes of PA were 
transformed for the t-tests and then back-transformed for values reported in the table. 


‘ Statistically significant at p < 0.05 


Table 6 shows the average supportive PA 
environment scores and minutes of weekly PA 
provided at schools based on the total number 
of Boston Foundation-funded PA programs in 
those schools. Similar to the findings in Table 
5, our analysis showed that supportive PA 
environment scores were higher on average 

in schools that had more Boston Foundation- 
funded programs. Statistical significance 
persisted when we ran this same analysis 
excluding high schools (p=0.01). It is again 
difficult to determine causal direction of these 
relationships given the cross-sectional nature 
of this data. 


There were no significant differences in 
minutes of PA provided based on number 
of PA programs. 


Table 7 shows the average number of total 
Boston Foundation-funded programs in 
buildings that have versus do not have various 
types of supports for children’s PA. On 
average, there was a significantly greater 
number of Boston Foundation programs 

in schools that offered staff professional 
development on PA or PE (1.1 programs on 
average) compared with those that did not 
offer such opportunities (0.4 programs, 
p=0.004). There was also a borderline- 
significant (p=0.08) difference in the average 
number of programs in schools with gym 
facilities (1.2 programs) compared to those 
without such facilities (0.8 programs). 


TABLE 6 
ASSOCIATIONS BETWEEN SCHOOLS’ MEAN SUPPORTIVE PA ENVIRONMENT SCORES AND 
MEAN MINUTES OF WEEKLY PA PROVIDED (IN RECESS, CLASSROOM BREAKS, ACTIVE LESSONS 
AND PROMOTIONAL ACTIVITIES) AND NUMBER OF TBF-FUNDED PA PROGRAMS, 2017-2018 


PA CATEGORY 


Supportive PA Environment Score 


NUMBER OF TBF PROGRAMS 


1 3+ 
(59) (N=17) 
9.4 11.1 12.4 12.9 0.0005* 


Min. of School PA Provided (Weekly) 195.8 


Data Source: BPS Profiles Data 


211.2 242.8 154.9 0.3000 


p-values are for results of analysis of variance (ANOVA) comparing mean values across groups. Due to skewness, the values for mean minutes of PA were 
transformed for ANOVA and then back-transformed for values reported in the table. 


t Statistically significant at p < 0.05 


TABLE 7 
MEAN NUMBER OF TBF-FUNDED PROGRAMS IN BOSTON PUBLIC SCHOOLS 
WITH AND WITHOUT PA SUPPORTS, 2017-2018 


HAVE BUDGET FOR | HAVE PE OR PA PROF. 
EQUIPMENT/ DEVELOPMENT _| HAVE pelea 
CURRICULUM OPPORTUNITIES 


Mean # of 
TBF PA 
Programs 


Data Source: BPS Profiles Data 


PRINCIPAL 
ENCOURAGEMENT FOR 
PA BREAKS 


CSPAP PLAN 
DEVELOPED 


p-values are for results of independent-samples t-tests of differences between means. 


i Statistically significant at p < 0.05 


School-Based Running Initiative 


Systematic Review 


From 2014 to 2019, the Boston Foundation 
supported implementation of Child- 
Obesity180's work to expand youth walking/ 
running clubs in Boston. The project helped 
public and charter schools and community 
centers implement free walking /running 
programs, with the goal of adding incremental 
PA. During its final year, the initiative reached 
80+ sites, with more than 10,000 students partic- 
ipating. During each program year, run club 
coaches completed surveys regarding program 
implementation and results. ChildObesity180 
published a peer-reviewed article using these 
survey findings to share learnings and best 
practices from the project to help other schools 
and districts to implement such initiatives."* 


Our run club surveys provided encouraging 
evidence about children’s PA and improve- 
ments in fitness, as reported by coaches; 
however, such self-reported data can be prone 
to bias. Our original protocol for this research 
(which had been planned for Spring 2020) 
included more rigorous, accelerometer-based 
evaluation of children’s PA in Boston run 

clubs and other Boston Foundation-funded 
programs. However, that protocol was 
discontinued due to school closures and 
limitations on human subjects research. When 
our new protocol was implemented in 2021, 
we no longer had a sufficient understanding 
of which run clubs were still in place to include 
run club programs in the study. 


As an alternative way of understanding the 
impact of run clubs on children’s PA and health, 
we conducted a systematic review of prior 
studies testing the effect of such running 
initiatives. Our two research questions were: 


1. What is the effect of youth running 
initiatives on children and adolescents’ 
objectively measured PA? 


2. What is the effect of youth running 
initiatives on children and adolescents’ 
cardiorespiratory fitness (CRF)? 


The systematic review followed the 2020 
Preferred Reporting Items for Systematic 
reviews and Meta-Analyses (PRISMA) 
guidelines.”” Our search process yielded 8,330 
titles and abstracts to review. After inclusion 
and exclusion criteria were applied, the final 
sample included 17 articles on controlled 
intervention trials: 12 articles assessed only 
CRF outcomes, three studies measured both 
PA and CRF outcomes, and two only evaluated 
PA outcomes. 


The systematic review findings are currently 
being prepared for a peer-reviewed journal 
submission. Preliminary findings include the 
following: 


e@ Ten out of 15 CRF-related studies 
demonstrated a significant effect on CRF. 


e@ Three out five studies demonstrated 
statistically significant improvements 
in PA levels. 


e Asmall number of studies explored 
differences in effects for boys versus 
girls; some studies showed that boys 
had significantly greater increases in 
PA compared to girls, but changes in 
CRF-related outcomes were similar 
for boys and girls overall. 


@ Few studies explored subgroup differences 
along dimensions like race-ethnicity, 
socioeconomic status, or weight status; 
future research should address this gap. 
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Overall, this evidence suggests that school- 
based running programs, similar to those 
supported in BPS, may have a modest but 
significant positive effect for children’s PA and 
CRF. Additional studies are needed to explore 
whether those benefits differ based on child 
characteristics like sex, race-ethnicity, or weight 
status. ChildObesity180 plans to complete this 
review and submit it for review to a scientific 
journal by the end of 2021. 


Physical Activity Program Staff Survey 


OVERVIEW 


The Physical Activity Program Staff Survey 
(Appendix A) consisted of 22 questions, 
including items regarding the perceived gender 
distribution of child participants, amount of PA 
programming provided (e.g., weeks of 
programming, days per week, active minutes 
per session), and spaces where programming 
was provided, both pre-pandemic (Fall 2019) 
and during the pandemic (Fall 2020). It also 
included items regarding perceptions of the 
impact of COVID-19 on program delivery. The 
survey was approved by both the Tufts 


University Institutional Review Board and the 
BPS Office of Data Accountability (ODA). 


The ODA requires that school administrators 
provide approval before any study subjects 

in their buildings are recruited. Of the 32 
administrators approached, 23 (72%) provided 
approval; 19 coaches were not school employees 
and therefore did not require administrator 
approval. In February and March 2021, we 
recruited 42 PA program coaches and had a 
response rate of 62% (n=26). 


Table 8 provides descriptive statistics of the 
individuals who responded to the survey. 


TABLE 8 
CHARACTERISTICS OF STAFF WHO RESPONDED TO THE COACH SURVEY (N=26) 


CHARACTERISTIC % N 

PROGRAM 

BOKS 46 12 

Community Rowing, Inc. 12 3 

Playworks 38 10 

Sportsmen's Tennis and Enrichment Center 4 1 
TIME COACHING THE PROGRAM 

lyear 31 8 

2 years 23 6 

3 years 19 5) 

4+ years 2 # 
ROLE RELATIVE TO PROGRAM 

Program staff member 69 18 

Physical education teacher 12 3} 

Other school staff 12 3 

Other roles 8 2 
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Table 9 and Table 10 include data only for 
survey respondents who served as coaches 

in both 2019 and 2020. These tables compare 
coaches’ perceptions of the gender distribution 
of participants, the amount of programming 
provided, and program delivery spaces from 
pre-pandemic to during the pandemic. Given 
the survey’s small sample size and limited 
statistical power, we do not include results 

of tests of statistical significance. 


The perceived gender distribution of partici- 
pants was relatively stable, though girls were 
slightly underrepresented both before and 


during the pandemic. Nearly all dimensions 
of program dose decreased from 2019 to 2020, 
including number of weeks, number of days 
per week, and number of minutes per session. 
However, coaches perceived relative stability 
in the percentage of programming time that 
children were active, with coaches reporting 
that kids were moving about two-thirds of the 
time both before and during the pandemic. 

It is worth noting, however, that such coach- 
reported data on the proportion of time 
children were active is likely prone to bias. 


TABLE 9 
CHANGES FROM FALL 2019 (PRE-PANDEMIC) TO FALL 2020 (MID-PANDEMIC) IN COACH-REPORTED 
GENDER COMPOSITION OF PARTICIPANTS AND DOSE OF PA PROGRAMMING PROVIDED (N=17) 


ATTRIBUTE OF PROGRAMMING 


2019 MEAN (SD) 2020 MEAN (SD) 


PARTICIPANT GENDER 
% girls 41.1 (12.6) 38.6 (17.6) 
AMOUNT OF PHYSICAL ACTIVITY 
Weeks of programming 12.4 (5.2) 9.3 (6.2) 
Days per week 3.5 (1.4) 25 (1-9) 
Average minutes /session (N=16) 43.8 (11.3) Sor2i(21k0) 
Minutes active (N=16) 30.7 (11.8) 22.8 (15.8) 
% time active (N=14) AeA 23271) 67.7 (20.3) 
Total dose (weeks x days x active minutes) 1479 (793.5) 661.9 (605.4) 
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Table 10 shows changes in the spaces that 
were used for program delivery. Prior to the 
pandemic, the main spaces used included 
typical school spaces like gymnasiums, 
playgrounds, multipurpose rooms, blacktop 
areas, and classrooms. During the pandemic, 
delivery shifted mainly to distance options 
(e.g., remote delivery platforms and recorded 


video); to the extent programming was still 
held in schools, it was mostly in outdoor spaces, 
like playgrounds and blacktop areas. These 
findings emphasize the extent to which 
programs pivoted to new delivery models 
during the pandemic so they could continue 
serving children despite challenges like school 
closures. 


TABLE 10 
CHANGES FROM FALL 2019 (PRE-PANDEMIC) TO FALL 2020 (DURING PANDEMIC) 
IN SPACES/CHANNELS USED FOR PROGRAM DELIVERY (N=17) 


SPACE/CHANNEL 2019 % = 2020 % = 


Gymnasium 65% 11% 

Multipurpose room/cafeteria 53% 9 11% 2 
Blacktop area 47% 8 17% 3 
Grassy field 18% 3 0% 0 
Playground 65% 11 28% 5 
Classroom 35% 6 11% 2 
Trailers /mobile building 6% 1 0% 0 
Community center 6% 1 6% 1 
eee ee 2 mo : 
Recorded video 0% 0 22% 4 
Other 12% 2 11% 2 
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Table 11 shows the proportion of coach 
respondents who agreed with statements 
about the impact of the pandemic on various 
aspects of program delivery. Coaches generally 
recognized that the pandemic had decreased 
program reach, with only about a quarter 

of participants agreeing that the program 
reached as many children as it would have 
without the pandemic. 


Fewer than half of respondents agreed that 
they were able to deliver programming as 

it was intended, and only about one in five 
agreed that the program was as effective at 
getting kids moving as it would have been 
without the pandemic. Although most coaches 


believed that they were able to deliver remote 
lessons successfully, only about a third agreed 
that children had the space and equipment 
they needed to participate in remote activities 
successfully. 


Only about half of participants agreed that 
the pandemic made them feel uncertain about 
how programming would be delivered in the 
coming months; this may be related to the fact 
that the survey was administered relatively 
late in the pandemic (early 2021), by which 
point programs may have reached a greater 
level of stability and predictability with 
program delivery. 


TABLE 11 


COACH PERCEPTIONS OF THE IMPACT OF THE COVID-19 PANDEMIC ON 
PHYSICAL ACTIVITY PROGRAM DELIVERY IN BOSTON PUBLIC SCHOOLS (N=26) 


% AGREE OR 
STATEMENT STRONGLY AGREE 


During the pandemic, program is able to reach as many children as it would have been without 039% 
the pandemic. ‘ 
During the pandemic, programming is as effective at getting kids physically active as it would 199% 
have been without the pandemic. i 
During the pandemic, Iam able to deliver program activities or lessons as they were intended to 409 
be delivered. z 
The pandemic makes me feel uncertain about how I will be delivering programming in the 58% 
coming months. 3 
During the pandemic, I have been able to successfully deliver program activities or lessons 81% 
remotely. : 
During the pandemic, our participants had the physical space and equipment (e.g., technology) 31% 
to engage with activities or lessons remotely. i 


Physical Activity Program Leadership Survey 


OVERVIEW Program leaders were recruited to participate 
The Physical Activity Program Leadership in the survey in January and February of 2021. 
Survey consisted of 12 questions about the Eight leaders from participating PA programs 
number of Boston Public Schools adopting were recruited, with a response rate of 63% 
programming, student-level program reach, (n=5). At least one survey was completed for 
the number of program staff, and the amount each of the Boston Foundation-funded PA 

of programming delivered (e.g., number of programs. For one program, two leaders 
weeks, number of days/week) in a typical completed the survey, and the responses for 
program year. It also explored the impact of those leaders were averaged, so that only one 
COVID-19 on some of these items. The survey set of values were used for that program in 
instrument is included in Appendix B. our analyses. 


TABLE 12 


MEAN NUMBER OF BPS SITES AND STUDENTS SERVED AND NUMBER OF PAID PROGRAM STAFF 
FOR TBF-FUNDED PHYSICAL ACTIVITY PROGRAMS, 2015-2020 


2015-2020 
AVG. PER YEAR 
30 13 


Mean # of BPS sites served (SD and range) (24, 8-65) (26, 5-65) (16, 0-34) 
5,663 4,444 2,733 
Noite Aeieurte ea ceed (UENCE CW s3) | ava ainsi) | (yon isa IaUD (4,584, 09543) 
43 40 15 
Mean # of paid staff (SD and range) (59, 8-130) (60, 4-130) (18, 0-40) 


Table 12 shows the average number of BPSsites most of the academic year and most days per 
and students served on average from 2015 to week and each session averaging about an 
2020, as well as in Fall 2019 compared with Fall hour and a half in length. 

2020. On average, the programs had broad reach 


; ; i . TABLE 13 
in BPS during this time and had sizeable staffs MEAN PROGRAM DURATION, NUMBER OF DAYS 
to support program delivery. However, there PROGRAMMING WAS OFFERED EACH WEEK, 
was substantial drop-off from pre-pandemic to AND LENGTH OF PROGRAM SESSIONS, 2015-2020 


mid-pandemic in terms of schools and students DIMENSION OF DOSE MEAN (SD, RANGE) 


reached and number of paid staff. 
Mean weeks of program duration per 27 
: hool : 
Table 13 shows the mean program duration, cestaees Sa Eee) 
days per week programming was provided, Mean days per week program offered a o 
and length of program sessions from 2015 to é 
2020. On average, schools provided a high Mean length of program session in minutes (101 i 250) 


program dose, with programming offered for 
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Key Informant Interviews 


OVERVIEW 


To complement the quantitative data analyses 
conducted with secondary data and original 
survey data, key informant interviews were 
conducted with program leaders via Zoom 
to gain additional qualitative insights. These 
interviews explored perceptions around 
programs’ building- and child-level reach 
and effectiveness, implementation processes 
and barriers, and perceptions around the 
programs’ partnerships with the Boston 
Foundation. The interview script is included 
in Appendix C. All methods were approved 
by the Tufts University IRB. 


Two senior leaders from each of the four 
participating programs, all with director- or 
manager-level positions, were recruited to 
participate, and all eight leaders agreed. 
Interviews occurred in January 2021. 
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In emails prior to the interview and during 
the interview itself, we emphasized that results 
would be fully anonymized to encourage 
participants to provide candid responses. 
Upon completion of all interviews, interviews 
were transcribed. The transcriptions were 
reviewed independently by two researchers 
and themes were then determined by 
consensus. One researcher then re-reviewed 
the transcripts and coded them for those 
themes using NVivo. 


Tables 14-19 summarize key themes and 
representative quotations in five different 
areas: program adoption and reach, program 
delivery, barriers to implementation, program 
effectiveness, and the impact of programs’ 
relationship to the Boston Foundation. The 
numbers accompanying each quotation are 
codes for the interviewees. 


TABLE 14 


KEY THEMES FROM INTERVIEWS WITH PHYSICAL ACTIVITY PROGRAM LEADERS REGARDING 
PROGRAM ADOPTION AND REACH 


THEME REPRESENTATIVE QUOTATION 


During typical years, programs prioritize 
reaching schools with high levels of 
disadvantage and schools with limited 
existing programming. 


¢ Onaverage, about 80% of the students we serve are considered high 
need. And Massachusetts Department of Elementary and Secondary 
Education defines that as anything from economically disadvantaged 
to SEL or EL learner or special needs. (6) 


We want to go where we can make the most difference, especially 
where kids may have limited access to [programming]. A lot of times, 
that means that schools have a need to increase the physical activity 
because of whether it be rates of obesity in their community or their 
schools, or poverty, or lack of access to other programs. (1) 


Programs added almost no new schools 
during the pandemic, and there was 
substantial falloff in terms of returning 
schools 


¢ We have not had any new schools coming on board this year. (3) 
¢ I’dsay about half of the schools are still doing [Program]. (7) 


e A lot of what COVID has done has impacted our ability to deliver 
our services, which has meant we have fewer partners than we had 
previously. (5) 


Schools with which programs had a long 
history of engagement were more likely to 
continue adopting programs through the 
pandemic 


Schools that have been with [Program] for a long time, where we have 
established relationships with principals and with PE teachers, those 
are the schools where we've seen a lot of people still return to 
[Program]. (7) 

e We've been there so long that we're almost looked at as being part of 

a school structure. So, they incorporate us in conversations that they 
might not have included us in if we were a brand-new partner. (3) 


That’s probably the biggest pattern, to be honest, is that there’s just been 
a consistency of working with those schools for a long time. And so they 
trusted our ability to adapt this year and that they would want us to 
continue there in the future. (5) 


Due to the pandemic, there were 
substantial decreases in the number 
of children reached overall 


I think that the cuts that we made as an organization and the limits that 
we had to impose on the numbers of people in the building, those were 
the biggest impact on the number of kids served. (1) 


I’m going to say the long trend is, it’s reduced so greatly because so 
much of our work was within the school and being able to work on the 
school site and a lot of the additions like field trips. (2) 


e Inthe summer, typically, we have 400 people [in programming space], 
and this summer we had 40 people [in programming space]. (2) 


Through the applications, the 27 schools gave us just over 10,000 kids 
they expected to participate. Through our first round of grantee reports 
they told us 4,800 were participating, so, definitely less than 50% of 
what they thought would be. And just looking at their grant reports, 
from what they told us, a lot of itis just heavy screen time, getting kids 
on board, and finding the right way to incentivize them and bring up 
engagement seem to be part of the issues that they were seeing. (8) 


Children with special needs were 
particularly challenging to reach during 
the pandemic 


We serve kids with disabilities under the program too and for a lot of 
reasons, transportation and otherwise, those numbers are way off. 
So, kids with disabilities, I think were probably impacted the most... 
To expect students with disabilities to follow every code guideline in 
person—we heard from families that that was probably a lot. (1) 
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TABLE 15 
KEY THEMES FROM INTERVIEWS WITH PHYSICAL ACTIVITY PROGRAM LEADERS REGARDING 
PROGRAM DELIVERY 


THEME REPRESENTATIVE QUOTATION 


¢ We're working with a few non-school sites to provide a similar type 
of experience and we're starting to get some traction, but that took a 
tremendous amount of phone calls and approvals and permissions. (1) 


Programs transitioned from in-school ¢ In addition to just fundamentally changing our program, 
delivery to delivery in non-school sites implementation is shifting from an in-person provider working with 
or to virtual delivery the space of whatever size blacktop that school has, we are now just 


doing it completely differently. The bulk of the work needs to 
maximize that remotely in the virtual setting to make it a fun, safe, 
inclusive, and active space for kids. (6) 


e We're really trying to move away from the thought of prescribing these 
Programs provided flexibility with rigid lesson plans and really just providing resources for anyone to use 


implementation to ensure programming as they see fit. (7) 


could continue to be delivered ¢ This has forced us ina really good way to be much more creative and 
deliver new fresh content that’s relevant now. (8) 


¢ We lost the ability to do some of the kinds of things virtually that we 
did with our direct participants that we couldn’t do with school-based 
participants. You're trying to figure out who’s going to dial in today. 
Who's going to log in? How do you keep them engaged? They don’t 
have the same equipment in front of them. (3) 

Remote delivery did not fully replicate 


: ‘ ' ¢ There are pieces that we aren’t able to do ina virtual setting. Some 
the experience of in-person programming 


of the things around culture and community are just not as possible 
virtually. (5) 


¢ Kids are not always turning their cameras on.... [When] you're in 
person... you can see which kids are participating [and] who maybe 
needs a little extra support. It’s not always that easy over camera. (7) 


e We've done Facebook Live [programming], we've done the weekly 
games, we've done the drop-in sessions, a family night and I think 
those things are here to stay and we'll continue to do that. And we'll 
use the virtual space as a way to promote it and to have direct access 


to that. (5) 
Participants reported various program e That’s one thing that we hope to maintain is continuing to send home 
innovations, including remote delivery, resources that the kids can do to play at home with their families, and 
take-home kits, activities that don’t to continue to provide more and more resources to families so that 
require equipment, and new methods for they have that access too. (6) 
engaging families; these innovations are ¢ Our biggest win in terms of future implementation is the fact that 
likely to persist into the future we've created all these resources that remove barriers, they remove 


equipment, they removed time constraints, and so I think they'll be 
much easier for all schools to implement moving forward. (7) 


¢ We quickly pivoted to create more digital program activities that 
were geared towards individuals working in that hybrid remote 
environment. (8) 


TABLE 15 continues next page 
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TABLE 15 continued 


THEME REPRESENTATIVE QUOTATION 


Participants expressed overall 
confidence in their programs’ ability 
to rebound from the pandemic and 
implement programming successfully 


e [have every reason to believe that we will, if anything, have a broader 
reach than what we've had in the past because of the fact that there will 
be more schools that will be open to this kind of environment. (3) 


e I’mhighly optimistic. And the reason I feel that way is, I think people 
care about kids and making sure that they have the resources and tools 
and people in their lives that are going to help them physically, socially, 
emotionally. (6) 

e I’m very confident that we'll be able to continue with an improved 
impact and my hope is that we will continue to impact more schools, 
because of our increased flexibility. (7) 


After the pandemic, programs may 
have a particular need to train staff 
in trauma-informed delivery 


¢ We feel very confident in our ability to train coaches, hire coaches, put 
them in schools, but I do think there'll be challenges around mental 
health and trauma and things that are a result of COVID that we maybe 
haven't had to tackle before and potentially might need additional 
training for our staff on how to handle those things. (5) 


e I think there are severe supports needed for trauma and anxiety 
and depression that kids are facing ... So, if students have dealt with 
trauma, we're going to have to make sure we have a whole series on 
trauma-informed practices and how to modify our programming to 
support students in new ways. (6) 


TABLE 16 


KEY THEMES FROM INTERVIEWS WITH PHYSICAL ACTIVITY PROGRAM LEADERS REGARDING 
BARRIERS TO PROGRAM DELIVERY 


THEME REPRESENTATIVE QUOTATION 


Complicated COVID restrictions (e.g., 
capacity limits, sanitation protocols) 
reduced capacity and slowed delivery 


¢ Right now, with what's going on, more of the guidance and direction 
is coming from the district level than from the individual schools. So, 
we're sort of trying to adjust and figure out what that’s going to look 
like. (3) 

¢ Are they tested, are they not tested, are they going to be safe to the 
community? We can't bring in extra people and the equipment they're 
using and when we have to bring in equipment, everything being 
wiped down if other kids are going to use it. There are so many other 
pieces now to it that you never really thought about before. (4) 


COVID-related uncertainty made 
planning difficult 


¢ Because there’s just so much uncertainty, it seems like every time you 
create a plan and you get halfway through the planning process, there 
was a change in what’s going to be allowed. And so we sort of start back 
from square one. (3) 

¢ The uncertainty around how schools would actually be implemented 
this fall has decreased the number of school partners that we decided 
to partner with now. (6) 


Increased competition with academic 
priorities made it difficult to convince 
some schools to support program delivery 


e Rightly so, the emphasis of course is on academics. So I think it was 
hard enough to get people to recognize the importance of physical 
activity, of quality physical activity, when it was easy to do in the school 
system. Now, when you have the situation where you're trying to 
convince the school leadership to invest in physical activity, I think it’s 
even that much more challenging really to get the time allocated and 
the resources allocated for physical activity. (3) 


¢ Some schools more than others have been willing to let us pop into a 
class that might be technically during their academic time. Others have 
said, nope, we need academic hours during all of this and you're not 
allowed to pop into any of that at all. (5) 


TABLE 16 continues next page 
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TABLE 16 continued 


THEME REPRESENTATIVE QUOTATION 


Some students had limited access to 
technology, complicating remote delivery 


e Virtual learning is not working in low-income communities. That’s just 
my opinion. I think that there are very few schools and families that 
were able to pivot successfully consistently. (3) 


e The technology piece and essentially just rolling out getting kids the 
technology that they need to actually access programming was pretty 
slow. (5) 


Some programs experienced resistance 
to programming delivered via screens 


We've had some schools that have said we want to prioritize breaks as 
being away from a computer and consequently they didn’t feel like the 
additional virtual [programming] was going to be as beneficial for their 
school. (5) 


¢ We had a lot of schools that said we love [Program], we'd love to offer 
[Program] in person. However, the feedback we're getting from families 
and our teachers is that we want to maximize our on-screen time and 
allow our kids to have breaks off-screen and so for those schools, we 
had to hit pause. (6) 


TABLE 17 


KEY THEMES FROM INTERVIEWS WITH PHYSICAL ACTIVITY PROGRAM LEADERS REGARDING 
PROGRAM EFFECTIVENESS AND IMPACT 


Program leaders believe that their 
programs increase children’s physical 
activity both now and into the future 


¢ Weare increasing the percentage of kids that are meeting those 
benchmark goals of the amount of moderate to vigorous physical 
activity they're getting. (3) 


It is not just for the sake of [sport] that we do this, but we also want to 
make sure that students are getting the physical activity and exercise 
that they need and also developing healthy lifestyles because this is 
something that they can use in the future. (4) 


Program leaders believe that their 
programs confer benefits for school 
attendance and academic performance 


The basic steps that we always talk about is we want to help with high 
school graduation, and college acceptance, college attendance, and help 
college graduation...we have, I think, over 95% college attendance and 
asimilar rate of high school graduation. (2) 


Icertainly think having movement breaks and having kids get a chance 
to take a break and play and move is probably benefiting them 
academically as well. (5) 

e At[Program] schools, we actually see an increase in attendance rate 


because I think there’s that joy of play and that kids are feeling safe 
and included. (6) 


Program leaders believe that their 
programs confer social-emotional 
benefits and inter-personal connection, 
which may be especially important 
during the pandemic 


¢ We talked about promoting positive school culture, so decreasing 
bullying, increases in positive language you might hear on the 
playground, changing the way students resolve conflicts with 
each other by teaching conflict resolution. (5) 


¢ The social, emotional, and physical health benefits that [Program] 
can offer their school is a fundamental component. (6) 


¢ We know that in some regard [Program] is providing...a social 
component that we know is missing in virtual learning. (7) 
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TABLE 18 


KEY THEMES FROM INTERVIEWS WITH PHYSICAL ACTIVITY PROGRAM LEADERS REGARDING 
PROGRAM STAFFING AND BUDGET 


During the pandemic, programs 
experienced budgetary decreases due to 
reduced dollars from funders and from 
other revenue sources 


¢ Overall, our organization struggled because one other facet is revenue 


base to cover a lot of the other programming and that revenue didn’t 
come in. So, a lot of our projections were really twisted upside down. (2) 
There were some cuts in funding, mostly due to some of our other 
funders having to freeze their budget. (7) 


Flexible funding was essential to sustain 
organizational health through the 
pandemic 


The best thing has been for us the funders who have said, as long as you 
can keep serving the mission, you can be flexible. If you need to spend it 
on the electric bill, spend it on the electric bill. So that has helped us 
tremendously. (1) 


And then there were some funders, like the Boston Foundation, who 
released a lot of their restrictions on their grants. And so we were able to 
say we have this funding, we're still providing the programming, we're 
not spending the funding the same way. So we're able to use some of that 
funding to provide... to sort of augment what we're doing with staff to 
keep them whole, and to keep them engaged. (3) 


Programs experienced decreases in the 
number of staff due to budget cuts and 
other COVID-related issues 


The main impact that COVID has had on us is the massive reductions in 
staff. (1) 


We're struggling a little bit right now, we're down staff and not by choice. 
I think the pandemic has had varying impacts on people. There are some 
folks that have been devastated by the loss of revenue. There are other 
folks that have said, you know what, I was trying to decide whether to 
stop working, go back to school and maybe the decision was made for 
me. (3) 

So we went from having at least two [Program] trainers at each school to 
maybe one, sometimes two. And again, that's with a little more than half 
of the schools that we were impacting previously. (7) 
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TABLE 19 
KEY THEMES FROM INTERVIEWS WITH PHYSICAL ACTIVITY PROGRAM LEADERS REGARDING 
THE IMPACT OF THEIR RELATIONSHIP WITH THE BOSTON FOUNDATION 


THEME REPRESENTATIVE QUOTATION 


¢ I felt like they gave us a lot of support to pursue growing our program. 
Getting to work with more schools, they were a huge support. (2) 

¢ I would say that the Foundation has allowed us to provide access to 
healthy quality physical activity that would have been lacking for 


TBF funding enabled PA programs to hundreds if not thousands of students. (3) 

expand their overall reach, especially to ¢ The Boston Foundation funds us so that we’re able to absorb most of the 
schools and children with socioeconomic costs, and in a few cases we have stopped all the costs, and we come there 
disadvantage for free and it’s only through funding from the Boston Foundation. (4) 


¢ [think the funding that we received from TBF definitely helps offset 
costs for our schools...Particularly right now budgets are really tough 
with schools and so that funding is incredibly important as we look at 
how much can schools really afford to pay for programming like ours. (5) 


¢ The ability of a grant-making organization to give multi-year 
commitments is a huge relief. We wouldn't have probably been able to 


The multi-year commitment from TBF launch it to the scale that we did without knowing that we had some 

and flexibility of the funding provided long-term funders really supporting what we were trying to do. (1) 
critical stability in general, and ¢ We unfortunately had budget cuts due to COVID...we had very minimal 
particularly through the pandemic funding into the [Program], which goes directly to schools, so we did 


have to heavily rely on external partners like the Boston Foundation to 
help continue to provide that funding for us. (8) 


¢ Our work with the Boston Foundation has really helped us be more 
thoughtful and reflective about the practice, especially as we scaled up 
... Ithelped us both design and reflect on what going to scale meant for 


\Syoenaie’ alana spend hele amstha a and ... how effective we were being and how we might change things. 


them to be more strategic and innovative ; : : ; ' 
8 e Their continued support has allowed us to continue to innovate and find 


new ways to get kids active. They not only have supported our expansion 
and serving our schools, but also the creation of these new programs and 
new supports for students in school. (6) 
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Strengths and Limitations 


Some strengths of our study include a mixed- 
methods design encompassing data gathered 
from diverse stakeholders, including PA 
program leaders, coaches, and school admin- 
istrators and staff. In addition, despite school 
closures and restrictions on human subjects 
research, we were able to develop a feasible 
study design that tested novel and timely 
research questions. We believe that these 
analyses yielded a variety of useful findings 
for the Boston Foundation and participating 
programs. These findings also have the 
potential to contribute to the peer-reviewed 


research literature; for example, there are, to our 
knowledge, no prior studies that have evaluated 


the impact of the COVID-19 pandemic on 
multiple school-based PA programs. 


Limitations of our study include our reliance 


on self-reported data, which raises the potential 
for various types of bias (e.g., social desirability 


bias and recall bias). We also had small sample 
sizes for some components of the research, 
particularly the surveys and key informant 
interviews, though we believe these data 
nonetheless provide a useful understanding 
of these programs. This research drew upon 
validated survey instruments to the extent 
possible; however, particularly given our 
interest in exploring questions related to 
the COVID-19 pandemic, we had to develop 
some instruments or individual questions 
that were new and not validated. Finally, 
the cross-sectional nature of our data limits 
the potential to draw causal inferences. 
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These findings 
have the potential 
to contribute to 
the peer-reviewed 
research literature; 
there are no 
prior studies that 
have evaluated 
the impact of 
the COVID-19 
pandemic on 
multiple 
school-based 
PA programs. 


Opportunities 


The findings from this project reflect a number 


of potential opportunities for the future: 


@ Boston Foundation-funded PA programs 
have broad, equitable reach in BPS and 
appear to have a positive impact on 
children’s PA and general well-being. 
Although program leaders expressed 
general optimism for their ability to 
rebound from COVID, they suffered 
significant cuts in budgets and staffing 
as well as decreases in the amount of 
programming they were able to provide. 
The programs may need supports to 
ensure they are able to fully return 
to normal operations. 


e@ Our findings showed that Boston 
Foundation-funded programs tend to 
reach schools with environments that are 
supportive of PA more generally. It may 
be that such schools are more responsive 
to adopting PA programs. However, 
finding ways to reach schools that have 
less supportive environments might 
help ensure more even distribution of 
PA opportunities across the BPS district. 


@ Survey data suggested that, overall, 
participating coaches perceived girls to 
be slightly under-represented in these 
PA programs both before and during 
the pandemic. These data should be 
interpreted somewhat cautiously given 
the small sample size and reliance on 
self-report; furthermore, our limited 
sample size constrained our ability to 


explore whether this same pattern held true 
for all four programs individually. Still, 
programs may benefit from examining 
the gender distribution of participants 
and ensuring they are equitably engaging 
girls. This is particularly important given 
research evidence showing that girls tend 
to be less physically active than boys on 
average and may therefore have particular 
need for these types of programs. 


Survey data and key informant interviews 
showed that PA programs innovated 
during the pandemic with developments 
like remote delivery models, take-home 
kits, equipment-free activities, and 

family engagement strategies. These 
innovations appear to have potential 

to persist into the future and to reach 
children and families in novel ways. 


Programs may benefit from securing new 
sources of long-term, flexible funding. 
Such funding both enables innovation in 
normal times and supports organizational 
health through shocks like the COVID-19 
pandemic. Grantmaking organizations 
should be aware of the critical importance 
of such funding for programs like these. 


Few studies have explored whether PA 
programs equitably impact children 
along dimensions like race-ethnicity, 
socioeconomic status, and weight 
status; future research exploring such 
equity-related questions would make a 
valuable contribution to the literature. 
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Appendix A: 


PA Program Staff Survey 


1. What is your role relative to [PA program name]? 
C) [PA Program name] staff member 
C) Physical education teacher 
C1) Classroom teacher 
C) Other school staff 
O) Parent 
C) Other (please explain) 


2. Including the current academic year, how many years have you led [specific PA 
program]? 
C1) 1 year (this is my first year) 
C) 2 years 
1) 3 years 
O) 4years 
C) Sor more years 


[Skip logic: If question 2 = 1 year, skip to Question 10) 


For the next items, please reflect back on delivery of [program name] programming in Fall 


(September-December) 2019. 


3. In Fall 2019, approximately what proportion of Boston Public School [program name] 
participants were girls? 
CX) 0-20% 
CO) 21-40% 
O 41-60% 
C) 61-80% 
C) 81-100% 


4. Please mark all spaces/channels that you used to deliver [program name] programming 
to Boston Public School students in Fall 2019 (check all that apply) 

Gymnasium 

Multipurpose room/cafeteria 

Blacktop area 

Grassy field 

Playground 

Classroom 

Trailers/mobile building 

Community center 

Live, remote delivery platform (e.g., Zoom, Google Hangouts) 

Recorded video 

Other [please explain, open field] 


ak fa se Gs | 
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On average, across all sites with which you worked, how many weeks was the program 
active in fall 2019? [Numeric field with a maximum of 20 weeks] 


On average, during the weeks your program was active, how many days per week did 
your program meet in Fall 2019? [Numeric field with maximum of 7 days] 


On average, what was the scheduled length of a typical program session in Fall 2019? 
scheduled minutes 


Questions 8. And 9. Ask about the average actual minutes reported in question 8. The 
sum of minutes for these items should equal the number of minutes reported in 
question 8. 


In Fall 2019, during a typical program session, think about how long most students were 
typically physically inactive, such as while receiving instructions or waiting for a turn. 
Estimate the number of total minutes students were typically lying down, sitting, or 
standing. minutes per session (Provide the average) 


In Fall 2019, during a typical program session, think about how long most students were 
physically active at least to the intensity level of a moderately paced walk. Estimate the 
number of total minutes students were engaged in moderate to vigorous physical 
activity. minutes per session (Provide the average) 


For the next items, please reflect on delivery of [program name] programming in Fall 


(September-December) 2020. 


10. 


11. 


In Fall 2020, approximately what proportion of Boston Public School [program name] 
participants were girls? 

O 0-20% 

O) 21-40% 

CO 41-60% 

O) 61-80% 

C) 81-100% 


Please mark all spaces/channels that you used to deliver [program name] programming 
to Boston Public Schools students in Fall 2020 (check all that apply) 


CO Gymnasium C) Community center 

C] Multipurpose room/cafeteria C) Live, remote delivery platform 
C) Blacktop area (e.g., Zoom, Google Hangouts) 
OC) Grassy field C) Recorded video 

1] Playground C) Other [please explain, open 
1 Classroom field] 

O) Trailers/mobile building 
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12. On average, across all sites with which you worked, how many weeks was the program 
active in fall 2020? [Numeric field with a maximum of 20 weeks] 


13. On average, during the weeks your program was active, how many days per week did 
your program meet in Fall 2020? [Numeric field with maximum of 7 days] 


14. On average, what was the scheduled length of a typical program session in Fall 2020? 
scheduled minutes 


Questions 15. And 16. Ask about the average actual minutes reported in question 9. The 
sum of minutes for these items should equal the number of minutes reported in 
question 9. 


15. In Fall 2020, during a typical program session, think about how long most students were 
typically physically inactive, such as while receiving instructions or waiting for a turn. 
Estimate the number of total minutes students were typically lying down, sitting, or 
standing. minutes per session (Provide the average) 


16. In Fall 2020, during a typical program session, think about how long most students were 
physically active at least to the intensity level of a moderately paced walk. Estimate the 
number of total minutes students were engaged in moderate to vigorous physical 
activity. minutes per session (Provide the average) 


The next questions relate to your perceptions of how the COVID-19 pandemic affects 
implementation of [program name] programming for Boston Public School students. Please 
indicate your level of agreement with each statement. 


[Five-point Likert, scale strongly disagree to strongly agree or (NA) if not relevant to program] 


been without the pandemic. 

18. During the pandemic, [program name] programming is as effective at getting kids 
physically active as it would have been without the pandemic. 

19. During the pandemic, | am as able to deliver [program name] activities or lessons as 
they were intended to be delivered. 

20. The pandemic makes me feel uncertain about how | will be delivering [program name] 
programming in the coming months. 

21. During the pandemic, | have been able to successfully deliver [program name] activities 
or lessons remotely. 

22. During the pandemic, our participants had the physical space and equipment (e.g., 
technology) to engage with activities or lessons remotely. 
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Appendix B: 


PA Program Leadership Survey 


Number of adopting schools 


On average, from 2015 to now, in approximately how many Boston Public Schools was 
[program name] programming provided each year? [open field] 


From September to December 2019, in approximately how many Boston Public Schools was 
[program name] programming provided? [open field] 


From September to December 2020, in approximately how many Boston Public Schools was 
[program name] programming provided? [open field] 


B. Child reach 


On average, from 2015 to now, approximately how many Boston Public Schools students 
have participated in [program name] each year? [open field] 


Approximately how many Boston Public Schools students participated in [program name] 
from September to December 2019? [open field] 


Approximately how many Boston Public Schools students participated in [program name] 
from September to December 2020? [open field] 


Staff and volunteers 


7. On average, from 2015 to now, approximately how many paid [program name] staff 


D. 


(including all adults who received payments or stipends) led [program name] programming 
in Boston Public Schools each year? [open field] 


Approximately how many paid [program name] staff led [program name] programming in 
Boston Public Schools from September to December 2019? [open field] 


Approximately how many paid [program name] staff led [program name] programming in 
Boston Public Schools from September to December 2020? [open field] 


Program dose 


For the next items, please provide responses about your programming as it was typically 
delivered in Boston Public Schools between 2015 and 2019, prior to the COVID-19 pandemic. 


10. For a typical school site, how many weeks was the [program name] active during the school 


year? [Numeric field with a maximum of 35 weeks] 


11. For a typical school site, during the weeks [program name] was active, how many days per 


week did your program meet, on average? [Numeric field with maximum of 7 days] 


12. For a typical school site, what was the scheduled length of a typical program session, on 


average? scheduled minutes 
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Appendix C: 


PA Program Leadership Key Informant 
Interviews Script 


Hi, this is colleague from Tufts University. 
My colleague is also on the call and will be assisting me today. 


Before we begin, can you please confirm that you have received and read through 
the informed consent document? 


Do you consent to participate in this study? 


Do you agree to be audiotaped? (Only continue if answer to both questions is yes) 


Thank you for taking the time to speak with me about your perceptions of [PA program 
name] and how the COVID pandemic has influenced programming. We understand how busy 
your schedule can be and appreciate your generosity with your time. 


Before we begin, I’d like to briefly reiterate some important points from the consent form. 


Your feedback today will help TBF and other individuals and organizations understand how 
school-based physical activity programs influence children as well as the impact of the 
pandemic on program delivery and outcomes. 


General feedback from this interview and interviews with other representatives will be 
summarized. That summary feedback will be provided to the Boston Foundation. We may 
also use those summarized feedback in research publications to help inform policy and 
practice more broadly. 


We’re taking notes and recording the session because we want to make sure we don’t miss 
any of your comments. Please feel free to opt-out of any questions you may not want to 
answer. You also have the right to stop participating in the interview at any time. 


| expect the interview to last approximately 30 to 45 minutes. Feel free to let me know if you 
need to pause or take a break at any time. 


Do you have any questions? Okay, let’s get started. 
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Introductory Questions: 


1. How long have you worked with [program name]? Can you tell me about the nature 
of your work with the organization? 
2. If you had to summarize your organization’s experience with the Boston Foundation 
in one sentence, what would you say? 
¢ Probe: How has funding from TBF impacted your ability to reach children 
and schools or changed the quality of programming provided? 


Topic Area 1: Impact of COVID on general program operations 

First, we’d like to discuss how the COVID epidemic has impacted your general program 

operations. 

1. Could you tell me about any ways in which COVID-19 has impacted your program budget 
this year? What have been the reasons for those changes? 

e Probes: more competitive funding climate, needing to divert funding to other 
priorities 

2. Could you explain how the pandemic has impacted the program administrators, including 
staff and any other adults who receive payment/stipends? For example, have the number 
or types of people who lead programming changed? 


Topic Area 2: Program adoption and maintenance 
Next, I’d like to talk about the Boston schools that have been involved with [program name] 
delivery and how that’s changed because of COVID. 
1. Tell me about the types of Boston Public Schools where [program name] has offered 
programming in recent years and the types of children that typically participate. 
Are there types of communities or individual children that you generally prioritize? 
2. To what extent have schools that were engaged with [program name] programming in the 
2019-20 academic year maintained their programming this year? 
a. Are there any patterns in terms of the types of schools that have maintained or 
not maintained programming? 
b. What factors do you think explain why schools have maintained or not maintained 
programming? 
3. To what extent have new schools newly adopted [program name] programming this year? 
Could you talk about why that might be? 
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Topic Area 3: Program reach 
Next, I’d like to talk about the children that [program name] reaches and whether that’s 
changed because of COVID-19. 
1. Could you tell me about any ways in which the number of Boston public school children that 
you’re reaching changed during the pandemic? 
a. What are your thoughts on why your overall reach has changed? 
i. Probes: school district or public health policies, changes in staffing on 
volunteer capacity, limitations around facilities 
2. Tell me about any ways in which the types of schools or children you’re reaching has 
changed. 
a. Probes: differences by gender, socioeconomic status, race-ethnicity, other factors? 
b. What are your thoughts on why the types of children you’re reaching have changed? 


Topic Area 4: Program implementation and effectiveness 

Now | want to ask you about your thoughts on how [program name] is being implemented and 

its effectiveness in promoting physical activity or other outcomes in participating children. 

1. First, can you share a bit about what program implementation has looked like for [program 
name] this year? 

a. How has implementation changed? Is your organization able to implement 
programming the way it’s intended to be delivered? 

b. Have you introduced any new implementation approaches or innovations that you 
might maintain into the future, even after the pandemic has passed? 

2. In typical times, what are the main ways in which you think [program name] affects 
participating children in Boston Public Schools? What types of outcomes do you think your 
program usually generates? 

3. Tell me about any changes in program effectiveness for children that have been able to 
continue participating in school since the pandemic. Has program effectiveness changed? 
How so? 


Topic Area 5: Recovery from the Pandemic 

1. How confident do you feel about [program name] ‘s ability to return to normal after the 
pandemic has come under control? What makes you feel that way? 

2. What types of resources or supports would help [program name] return to normal after the 
end of the pandemic? 


Those are all the questions | have for you. Is there anything else you’d like to share? 


Thank you for your participation. 
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Appendix D: 


Explanation of the Supportive Physical 
Activity Environment Variable 


This is an adapted version of the Physical Activity School Score (PASS),‘”? modified for this research study 
based on the variables available through the BPS Profiles data set. 


Criterion 1. Average weekly minutes of physical education provided across grade levels 
e 5 points: at least 150 minutes per week 
e 3>points: 90 to 149 minutes per week 
* 1point: 60 to 89 minutes per week 
e Opoints: fewer than 60 minutes per week 


Criterion 2. Physical education teacher is certified, licensed, or endorsed by the state to teach physical 
education in elementary, middle, or high school. 

e 1point: Yes 

e Opoints: No 


Criterion 3. Average minutes of weekly recess provided across grade levels 
5 points: at least 100 minutes per week 

4 points: from 80 to 99 minutes per week 

1 point: from to 60 to 79 minutes a week 

0 points: fewer than 60 minutes per week 


Criterion 4. Recess is supported by Playworks to promote PA 
e point: Yes 
e Opoints: No 


Criterion 5. The school provides opportunities for students to participate in daily physical activity during 
the school day through: (a) classroom activity breaks; (b) before school programs; (c) after school 
programs 

e 3 points: all three of the above 

* 2 points: two of the three above 

® 1point: one of the three above 

e Opoints: none of the three above 


Criterion 6. The school has a transportation liaison to facilitate communication regarding district efforts 
to promote safe, physically active travel to and from school 

e 1point: Yes 

e Opoints: No 


Criterion 7. Staff at the school are prohibited from withholding physical education or physical activity as 
punishment for students (e.g., such as incomplete work) 

e point: Yes 

e Opoints: No 


Criterion 8. The school has adequate facilities/space to support children's physical activity throughout 
the school day 

e 3>points: both indoor and outdoor physical activity facilities are available 

® 2 points: only indoor physical activity facilities are available 

* 1point: only outdoor physical activity facilities are available 

e Opoints: playspace/facilities are not available to support children being physically active 
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